Robust moving video object segmentation in the MPEG compressed domain

In this paper, we proposed a robust moving video object segmentation algorithm using features in the MPEG compressed domain. We first cluster the motion vectors and produce a motion mask. Then, a difference mask at 8 x 8 block size is extracted from the DC image by background subtraction method. Finally, the motion mask and the difference mask are combined conditionally to generate the final object mask based on a set of rules that are application specified and can be obtained with leaming or heuristic based methods. The experimental results show that this object segmentation scheme is more robust than those using DC image or motion vectors only. 1. INTRODUCTION Video object segmentation is very important for many video applications, e.g. video surveillance, video index, object-base coding, etc. With the advance of digital media compression technology, compressed videos compliant to MPEG U214 have become standard outputs in most of the state-of-the-art video application systems. However, traditional object segmentation techniques in the pixel domain cannot cater for the increasing demand of processing large amount of compressed videos efficiently: both fully decoding the compressed video and segmenting objects in the pixel domain demand high computation cost. Thus, fast algorithms to extract objects from the compressed video directly are desired. In general, video object segmentation in the MPEG compressed domain relies on two types of compressed features: motion vectors and DCT coefficients. Motion vector can be viewed as a coarse approximation of optical flow in the MPEG compressed domain. Segmentation using motion vectors usually employs clustering methods to extract regions with homogeneous motion characters [ 1, 21. DCT coefficients contain the transformed spatial information. Most existed segmentation algorithms using DCT coefficients exploit DC image and AC energy to locate the objects and measure the texture and edges [3,4]. However, only few researchers have investigated the algorithms to combine the information in the motion vector domain and DCT domain for object segmentation. Jamrozik and Hayes [5] proposed an algorithm to discriminate objects by segmenting the DC image with leveled watershed technique and merging regions based on their motions similarities. Eng and Ma exploit their maximum entropy fuzzy clustering algorithm to classify the motion vectors and DC coefficients [6] into homogeneous regions. In this paper, we will present a robust object segmentation algorithm using motion vector and DC coefficients jointly. An overall diagram of the proposed algorithm is illustrated in Figure I. (b) Figure I: …